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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 
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2a)KI This action is FII^AL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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6) 13 Claim(s) 1-6,10-23 and 28-30 is/are rejected. 
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DETAILED ACTION 
Drawings 

1. The drawings were received on November 28, 2003. These drawings are accepted. 

Information Disclosure Statement 

2. The information disclosure statement (IDS) submitted on November 28, 2003 was filed 
after the mailing date of the non-final rejection on July 22, 2003. The submission is in 
compliance with the provisions of 37 CFR 1 .97. Accordingly, the information disclosure 
statement is being considered by the examiner. 

Response to Arguments 

3. AppHcant's arguments with respect to claims 1,11, and 21 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Objections 

4. Claims 28-30 are objected to under 37 CFR 1.75(c), as being of improper dependent form 
for failing to further limit the subject matter of a previous claim. Applicant is required to cancel 
the claim(s), or amend the claim(s) to place the claim(s) in proper dependent form, or rewrite the 
claim(s) in independent form. The limitations of shifting the focus of expansion by an integer 
value, shifting the focus of expansion to improve the accuracy of a constraint, and shifting the 
focus of expansion non-iteratively to improve the accuracy of a constraint as recited by claims 
28, 29, and 30, respectively, are all recited by independent claim 1. 

Claim Rejections - 35 USC § 112 

5. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
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pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

6. Claims 1-6, 7-23, and 28-30 are rejected under 35 U.S.C. 1 12, first paragraph, because 
the specification, while being enabling for "allowing the focus of expansions (FOE) of each 
point to shift "independently" by some integer amount, (Sx, Sy)" on page 25, lines 5-8 of the 
specification, does not reasonably provide enablement for "shifting a focus of expansion of the 
points on the figure by an integer value to non-iteratively improve an accuracy of a constraint" in 
claim 1, lines 5-6 and corresponding portions of claims 1 1 and 21 . The specification does not 
enable any person skilled in the art to which it pertains, or with which it is most nearly 
connected, to make the invention commensurate in scope with these claims. 

The recitation of "shifting of the focus of expansion of the points on the figure by an 
integer value to non-iteratively improve an accuracy of a constraint" in independent claims 1,11, 
and 21 does not require that the shift be independent of the previous shift. There may be a non- 
iterative shift (which the examiner construes to be not limited by the area of an original focus of 
expansion), which is dependent of the previous shift. For example, by shifting a focus of 
expansion by a predetermined integer pixel distance, the shift is dependent on the location of the 
previous focus of expansion, but is also non-iterative because the shift is not constrained by the 
area of the first focus of expansion. However, the specification requires an independent shift in 
this embodiment. 

Claims 2-6, 10, 12-20, 22-23, and 28-30 are rejected as being non-enabled for depending 
from a non-enabled antecedent base claim. 

7. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 
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8. Claims 1-6, 7-23, and 28-30 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Referring to claim 1,11, and 21, the step of "shifting a focus of expansion of the points 
on the figure by an integer value to non-iteratively improve an accuracy of a constraint" is 
indefinite. On page 24, line 12 to page 25, line 4 of the specification, the Applicant explains that 
the iterative estimation approach of first-order Taylor-series expansion used in brightness and 
depth constraints has several drawbacks. Then, on page 25, lines 5-16 of the specification, the 
Applicant introduces a method to improve the accuracy of the constraints "without iteration" by 
allowing the focus of expansions of each point to shift "independent" by some integer amount. 
Finally, on page 26, lines 1-9 of the specification, the Applicant explains that selecting the 
integer amounts (Sx, Sy) could be done within the traditional iterative motion estimation 
fi-amework. The Applicant explains that the motion is re-estimated according to the constraint 
equations, resulting in hierarchical and iterative processing. Therefore, it is not understood 
whether the determination of the integer values is non-iterative (as claimed) or iterative (as 
explained on page 26, lines 1-9) in light of the specification. 

Claims 2-6, 10, 12-20, 22-23, and 28-30 are rejected as being indefinite for depending 
from an indefinite antecedent base claim. 

Note: The uncertainties in this claim rises to the level of a 35 U.S.C. 1 12 rejection 
because of the inconsistency between the claimed subject matter and the specification (See 
MPEP 2173.03). 
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Claim Rejections - 35 USC § 103 



9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claims 1-3, 6, 1 1, 16, and 28-30 are rejected under 35 U.S.C, 103(a) as being 
unpatentable over Sabata et al (B. Sabata and J. Aggarwal. Estimation of motion from a pair of 
range images: A review. CVGIP, 54(3): 309-324, Nov. 1991.) in view of Nakajima et al (U.S. 
Patent No. 5,521,633). 

Referring to claim 1, 

i. Obtaining dense range data that describes the distance of points on the figure from 
a reference is explained by Sabata et al on page 310, second column, third paragraph. 
Sabata et al explain that the range image is the 3D coordinate of each point (or pixel) 
sensed. A 3D coordinate inherently includes both the horizontal and vertical coordinates 
of the object in space and also the distance of the object from the image sensor. 

ii. Shifting a focus of expansion of the points on the figure by an integer value to 
non-iteratively improve an accuracy of a constraint is not explicitly explained by Sabata 
et al. However, Nakajima et al illustrate in figure 10(a) and 10(b) that a focus of 
expansion is shifted by an integer value on a figure in order to non-iteratively improve an 
accuracy of a constraint. The Examiner construes a focus of expansion to be a point and 
it corresponding search window on an original figure for which the position (or pose) in a 
next figure is being determined. In figure 10(a) of Nakajima et al, the focus of expansion 
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is point P and its corresponding window is Wi. The Examiner construes a non-iterative 
shift to be a shift that is not hmited by the area of the window of the original focus of 
expansion, hi figure 10(b), Nakajima et al clearly illustrate that the focus of expansion 
(P) is shifted in a raster fashion of a second figure, which is non-iterative. Nakajima et al 
explain that for each shift, a correlation value is determined in column 1, lines 26-42. 
The correlation equation of Nakajima et al is a fiinction of the coordinates of the pixels of 
the first focus of expansion and the shifted focus of expansion, as therefore, corresponds 
to an integer shift within each figure. Once the original focus of expansion has been 
shifted across the entire second figure, the focus of expansion with the highest correlation 
value is determined to be the next pose of an object, hi figure 10(b), the second focus of 
expansion corresponds to point Q and its window W2. Nakajima et al also explain using 
range data in order to improve the detection of an object in column 1, lines 43-48. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to shift the focus of expansion of the points on a figure by an integer 
value using the system of Nakajima et al to non-iteratively improve the accuracy of the 
constraint in the system of Sabata et al because determining the motion of a point from all 
the points on a subsequent frame would improve the accuracy of motion detection and 
pose estimation. 

iii. Processing the dense range data to estimate the pose of the figure is explained by 
Sabata et al on page 310, second column, third paragraph. Sabata et al explain that the 
motion of a rigid object in an image is estimated, corresponding to the pose of the object 
("pose determination" on page 309, first column, second paragraph). 
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Referring to claim 2, the dense range data being processed in accordance with a set of 
depth constraints to estimate the pose is explained by Sabata et al on page 312, first column, first 
and second paragraphs. Sabata et al explain on page 311, first column, forth paragraph that the 
pose estimation is determined by both a rotational component (R) and a translational component 
(T). Sabata et al then provide constraints to simplify the calculation of the rotation matrix to 
estimate the rotational component of the pose estimation. 

Referring to claim 3, the depth constraints being linear is explained by Sabata et al on 
page 311, first column, forth paragraph. Sabata et al explain that the rotational component of the 
pose estimation, which is constrained corresponding to claim 2, is a linear transform. 

Referring to claim 6, the dense range data being compared with an estimate of pose to 
produce an error value and iteratively revising the estimate to minimize the error is explained by 
Sabata et al on page 311, first column, second and third paragraph. Sabata et al explain 
determining the error (Best) of the motion estimations and minimizing the error. 

Referring to claim 11, 

i. Obtaining dense range data that describes the distance of points on the figure from 
a reference corresponds to claim li. 

ii. Shifting a focus of expansion of the points on the figure by an integer value to 
non-iteratively improve an accuracy of a constraint corresponds to claim lii. 

iii. Processing the dense range data in accordance with a set of linear depth 
constraints to estimate the pose of the figure corresponds to claim 2. 
Referring to claim 16, 



Application/Control Number: 09/520,964 Page 8 

Art Unit: 2621 

i. The estimate of the pose of the object including an estimate for each of the 
orientation and translational positions of the object is explained by Sabata et al on page 
310, first column, forth paragraph. Sabata et al explain estimating the rotational 
component, corresponding to the orientation, and the translational component of the 
motion of an object. 

ii. Decoupling the estimate of orientation from the estimate of the translational 
position is explained by Sabata et al on page 311, second column, first paragraph. Sabata 
et al clearly illustrate that the position of point on a succeeding fr-ame is represented by 
the rotational component and the translational component independently. 

Referring to claim 28, the focus of expansion being shifted by an integer value 
corresponds to claim li. 

Referring to claim 29, shifting the focus of expansion improving an accuracy of a 
constraint corresponds to claim li. 

Referring to claim 30, shifting the focus of expansion non-iteratively improving an 
accuracy of a constraint corresponds to claim li. 

11. Claims 4-5, 10, 12-15, 17, and 21-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Sabata et al in view of Nakajima et al, and further in viev^ of Bregler et al 
(Bregler, C; Malik, J. Tracking people with twists and exponential maps. 1998 IEEE Computer 
Society Conference on Computer Vision and Pattern Recognition, 23-25 June 1998. Page(s): 8 - 
15.). 

Referring to claims 4, 10, and 15, obtaining brightness data in accordance with a set of 
linear brightness constraints to estimate the pose is not expUcitly explained by Sabata et al or 
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Nakajima et al. Sabata et al do explain that the intensity (corresponding to the brightness) of an 
image is widely used in the art to determine the motion of an object on page 309, first column, 
second paragraph to page 309, second column, first paragraph. However, Sabata et al do not 
explain linear brightness constraints on the intensity data. Bregler et al do explain linear 
constraints on the brightness data on page 9, first column, forth through seventh paragraphs. 
Bregler et al explain constructing an over-constrained set of equations for the motion model, 
assuming that the motion is linear. By applying the linearly constrained brightness data into the 
motion data determined by the range data of Sabata et al and Nakajima et al, the motion of an 
object through a series of frames will be estimated more accurately. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to use 
brightness data in accordance with a set of linear brightness constraints to better determine the 
pose of an object. 

Referring to claim 12, the object being articulated is not explicitly explained by Sabata et 
al or Nakajima et al. However, Bregler et al do illustrate estimating the pose of an articulated 
object in figure 2. The system of Sabata et al and Nakajima et al models the motion of an object 
by the rotational and translation components of the object's movement. Thus, an articulated 
object could be efficiently tracked in the system of Sabata et al and Nakajima et al. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
track an articulated object using the system of Sabata et al. 

Referring to claims 5 and 13, the depth constraints being represented by twist 
mathematics is not explicitly explained by Sabata et al or Nakajima et al. However, Bregler et al 
do explain using twist mathematics in order to estimate the pose of a figure on page 8, second 
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column, first paragraph. Bregler et al explain that using twist mathematics in motion estimation, 
such as the estimation of Sabata et al and Nakajima et al, greatly simplifies the estimation and 
leads to a more robust tracking result on page 8, second colunm, second paragraph. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
use twist mathematics as suggested by Bregler et al in the motion estimation system of Sabata et 
al and Nakajima et al in order to perform the motion estimation and object tracking more 
robustly. 

Referring claim 14, 

i. Mapping the parameters that describe the rotation and translation of the object to 
a set of linear parameters is explained by Sabata et al on page 311, second column, 
second paragraph. Sabata et al explain that the rotation and translation components of the 
object movement are characterized by individual matrices. 

ii. Solving for the depth constraints is explained by Sabata et al on page 312, first 
column, first and second paragraph. The rotational depth constraints are explained by 
Sabata et al and used to estimate the object motion. 

iii. Re-mapping back to the original parameters to provide a pose estimate is 
explained by Sabata et al on page 313, second column, fifth paragraph to page 314, first 
column, first paragraph. Sabata et al explain that the parameters which determine the 
rotational and translational components of the object are used in order to solve for the 
motion of the object. 

Referring to claim 17, the reference comprising a location on the object and the pose 
being estimated fi'om the positions of points on the object relative to the locations is explained by 
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Sabata et al on page 310, second column, sixth paragraph to page 311, first column, second 
paragraph. Sabata et al explain determining the current position of an object (P') from the 
previous position of the object (P) after the determined transformation of the object. 
Referring to claim 21, 

i. Obtaining dense brightness data for pixels in each of the video images is 
explained by Sabata et al on page 309, first column, second paragraph. 

ii. Obtaining dense range data for pixels in each of the video images corresponds to 
claim li. 

iii. Determining an initial pose for the object in one of the video images is explained 
by Sabata et al on page 311, first column, forth paragraph by p, which is the initial point 
on an object undergoing a transformation. 

iv. Estimating changes in at least one of the translational position and rotational 
orientation of the object for successive images on the basis of the brightness data and 
range data, shifting a focus of expansion of the points on the figure by an integer value to 
non-iteratively improve an accuracy of a constraint to estimate the pose of the object in 
successive images corresponds to claims lii, liii, and 4, wherein the pose is determined 
using the range data and the brightness data. 

Referring to claim 22, the object being articulated corresponds to claim 12. 

Referring to claim 23, the estimates being obtained by means of linear constraint 
equations that are applied to the brightness data and the range data corresponds to claims 4 and 3, 
respectively. 
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12. Claims 18-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Sabata et al 
in view of Nakajima et al, and further in view of Dubuisson- Jolly et al (U.S. Patent No. 
6,195,445). 

Referring to claim 18, the pose of an object being estimated for each image in a sequence 
of images and selecting a rigid translation value for each point on the object from one image to 
the next is not expHcitly explained by Sabata et al or Nakajima et al. However, Dubuisson- Jolly 
et al do illustrate rigid translation values for an articulated object in figure 4 by segment lengths 
<Di to O7 and explain polyline contours in column 6, lines 49-65. Dubuisson- Jolly et al explain 
that the polyline contours are tracked in an image sequence using an initial polyline contour in an 
input frame in column 6, lines 66 to column 7, line 10. By using the rigid translation value 
method of Dubuisson-Jolly et al, the processing required to track the motion of an articulated 
object would be reduced in the system of Sabata et al and Nakajima et al. Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made to use a 
rigid translation value for each segment of an articulated object in order to track the motion of an 
object more efficiently. 

Referring to claim 19, the rigid translation value being an integer value corresponds to 
claim 18, wherein C>i to may represent integer values of the rigid translation. 

Referring to claim 20, the rigid translation values being different for different points on 
the object corresponds to clam 18, wherein each segment, D, has a specific and different rigid 
translation value O. 
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Conclusion 



13. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi"om the maihng date of this action. In the event a first reply is filed within TWO 
MONTHS of the maihng date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated fi:-om the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS fi:'om the mailing 
date of this final action. 

14. Any inquiry concerning this communication or earlier communications fi*om the 
examiner should be directed to Hussein Akhavannik whose telephone number is (703)306-4049. 
The examiner can normally be reached on M-F 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessfial, the examiner's 
supervisor, Leo H. Boudreau can be reached on (703)305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




